{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 问题\n",
    "假设我们有一些样本点$\\boldsymbol{X}$和对应的目标$\\boldsymbol{y}$，已知$\\boldsymbol{y}$可以由$f(\\boldsymbol{x}) = \\boldsymbol{w \\cdot x}+ b$加上噪音生成，求解函数$f$的参数$\\boldsymbol{w}$和$b$。\n",
    "\n",
    "### 求解\n",
    "&emsp;&emsp;首先定义$f(\\boldsymbol{x})$相对于$y$的损失函数\n",
    "\n",
    "$$L(f,y) = \\frac{1}{2}\\sum_{\\boldsymbol{x}} (f(\\boldsymbol{x})-y)^2 = \\frac{1}{2}\\sum_{\\boldsymbol{x}}(\\boldsymbol{w \\cdot x}+b - y)^2$$\n",
    "\n",
    "那么损失函数相对于参数$\\boldsymbol{w}$和$b$的导数分别为:\n",
    "\n",
    "$$\\frac{\\partial L}{\\partial \\boldsymbol{w}} = \\sum_{\\boldsymbol{x}} \\boldsymbol{x}(\\boldsymbol{w \\cdot x} + b - y)$$\n",
    "\n",
    "$$\\frac{\\partial L}{\\partial b} =  \\sum_{\\boldsymbol{x}} (\\boldsymbol{w \\cdot x} + b - y)$$\n",
    "\n",
    "根据梯度下降法的公式可以得到参数的更新公式：\n",
    "\n",
    "$$\\boldsymbol{w} = \\boldsymbol{w} - \\eta \\frac{\\partial L}{\\partial \\boldsymbol{w}}$$\n",
    "$$b = b - \\eta \\frac{\\partial L}{\\partial b}$$\n",
    "\n",
    "下面将使用Python的Numpy实现上述过程"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAD9CAYAAABazssqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X10XPV95/H3R4+2Jdv4YWTkBwKWHKjtYCc4rhtIoHnC\nsBhIt806uxvYUw4kB5om225baM/pyf7BJk2btKUtNCRhMd001E2axRCclrgJWVIeIijY2MZgsMEW\nfhAYsLGxbEnf/WN+csayZI0tae5I83mdM2eufvfema8ugz6+9/e781NEYGZmlakq6wLMzCw7DgEz\nswrmEDAzq2AOATOzCuYQMDOrYA4BM7MKVnQISKqW9O+SHkg/T5X0kKQX0vOUgm1vkbRV0hZJlxa0\nXyBpQ1p3myQN769jZman4lTOBD4PbC74+WZgXUTMA9aln5E0H1gJLACWA7dLqk773AFcD8xLj+VD\nqt7MzIakqBCQNBv4D8A3C5qvAlal5VXA1QXt90ZEZ0RsA7YCSyU1A5Mi4rHI36F2T8E+ZmaWgWLP\nBP4C+H2gp6BtRkTsSsu7gRlpeRawo2C7naltVlru225mZhmpGWwDSVcAeyPiSUmX9LdNRISkYfv+\nCUk3ADcANDQ0XHDeeecN10ubmVWEJ5988rWIyA223aAhAFwIXCnpcmAcMEnS/wH2SGqOiF3pUs/e\ntH07MKdg/9mprT0t920/QUTcCdwJsGTJkmhrayuiTDMz6yXp5WK2G/RyUETcEhGzI+Js8h2+/xoR\n/xVYA1ybNrsWuC8trwFWSqqXdA75DuAn0qWj/ZKWpVFB1xTsY2ZmGSjmTGAgXwZWS7oOeBn4JEBE\nbJS0GtgEdAE3RUR32udG4G5gPLA2PczMLCMq96+S9uUgM7NTJ+nJiFgy2Ha+Y9jMrII5BMzMKphD\nwMysgjkEzMwq2JgNgXse3c79z7yadRlmZmVtzIbAP/x8B//45M7BNzQzq2BjNgRaco28uPftrMsw\nMytrYzYEWpsaaX/zHQ4d6cq6FDOzsjWmQwDgpY6DGVdiZla+xmwItOTyIfBihy8JmZkNZMyGwNnT\nJ1Al2Op+ATOzAY3ZEKivqeasqRN8JmBmdhJjNgQg3y/gMwEzs4GN6RBoyTWy/bVDdHX3DL6xmVkF\nGtsh0NTIke4edrzxTtalmJmVpbEdAr0jhHxJyMysX2M6BFpTCGx157CZWb8GDQFJ4yQ9IekZSRsl\n/c/U/kVJ7ZKeTo/LC/a5RdJWSVskXVrQfoGkDWndbWmu4REzeUIt0xvrfSZgZjaAYuYY7gQ+HBFv\nS6oFHpHUOzfwn0fEnxVuLGk++QnpFwAzgR9JeneaZ/gO4HrgceBBYDkjPM9wa1ODzwTMzAYw6JlA\n5PX+Fa1Nj5NNTHwVcG9EdEbENmArsFRSMzApIh6L/MTG9wBXD638wfUOEy33uZTNzLJQVJ+ApGpJ\nTwN7gYci4vG06nOS1ku6S9KU1DYL2FGw+87UNist920fUS25Rg4c7qLj7c6Rfiszs1GnqBCIiO6I\nWAzMJv+v+oXkL+3MBRYDu4CvDldRkm6Q1CapraOjY0iv1ftFcr5pzMzsRKc0Oigi3gR+DCyPiD0p\nHHqAbwBL02btwJyC3Wantva03Le9v/e5MyKWRMSSXC53KiWe4BdfJOdvEzUz66uY0UE5SWek5fHA\nx4Dn0jX+Xp8Ank3La4CVkuolnQPMA56IiF3AfknL0qiga4D7hvF36Vfz5HFMqKv2CCEzs34UMzqo\nGVglqZp8aKyOiAck/Z2kxeQ7ibcDnwGIiI2SVgObgC7gpjQyCOBG4G5gPPlRQSM6MghAUn6WMY8Q\nMjM7waAhEBHrgff20/7pk+xzK3BrP+1twMJTrHHIWpsaeeyl10v9tmZmZW9M3zHcqyXXwK63DvN2\np6eaNDMrVBEh8IupJn1JyMysUEWFgPsFzMyOVxEhcNbUBqqr5HsFzMz6qIgQqKup4l3TJjgEzMz6\nqIgQANIwUd8wZmZWqGJCoLWpke2vHeSop5o0MzumYkKgJddIV0/wyr5DWZdiZlY2KiYE/EVyZmYn\nqpgQmJtrADxM1MysUMWEwKRxtcyYVO8zATOzAhUTAuARQmZmfVVUCLQ2NfKip5o0Mzum4kLg7c4u\n9h7wVJNmZlBhIdA7y5j7BczM8ioqBDxM1MzseBUVAk0T62msr/EwUTOzpJg5hsdJekLSM5I2Svqf\nqX2qpIckvZCepxTsc4ukrZK2SLq0oP0CSRvSutvSXMMlI4mWpkafCZiZJcWcCXQCH46IRcBiYLmk\nZcDNwLqImAesSz8jaT6wElgALAduT/MTA9wBXE9+8vl5aX1JteQafCZgZpYMGgKR1/tXszY9ArgK\nWJXaVwFXp+WrgHsjojMitgFbgaWSmoFJEfFY5Mdo3lOwT8m0NjWyZ38n+w8fLfVbm5mVnaL6BCRV\nS3oa2As8FBGPAzMiYlfaZDcwIy3PAnYU7L4ztc1Ky33bS6p3hNBLvmnMzKy4EIiI7ohYDMwm/6/6\nhX3WB/mzg2Eh6QZJbZLaOjo6hutlAY8QMjMrdEqjgyLiTeDH5K/l70mXeEjPe9Nm7cCcgt1mp7b2\ntNy3vb/3uTMilkTEklwudyolDuqsqROorZb7BczMKG50UE7SGWl5PPAx4DlgDXBt2uxa4L60vAZY\nKale0jnkO4CfSJeO9ktalkYFXVOwT8nUVlfxrmkNPhMwMwNqitimGViVRvhUAasj4gFJjwKrJV0H\nvAx8EiAiNkpaDWwCuoCbIqI7vdaNwN3AeGBtepRca66R5/ceyOKtzczKyqAhEBHrgff20/468JEB\n9rkVuLWf9jZg4Yl7lFZLUwMPbd7Dka4e6moq6n45M7PjVORfwNamRrp7gpdf9wghM6tsFRkCvcNE\n3TlsZpWuokPAncNmVukqMgQa6mtonjzOs4yZWcWryBCAfL+AzwTMrNJVbAjk5xv2VJNmVtkqNwSa\nGjl0pJtdbx3OuhQzs8xUbAi0eoSQmVnlhkBLUwPgEUJmVtkqNgRyjfVMGuepJs2sslVsCHiqSTOz\nCg4ByPcLbN3rewXMrHJVdAi0NDXy2tudvHXIU02aWWWq6BDoHSG01f0CZlahKjoEWpo8TNTMKltF\nh8CcKeOpq67iRXcOm1mFqugQqKmu4pzpDT4TMLOKVcwcw3Mk/VjSJkkbJX0+tX9RUrukp9Pj8oJ9\nbpG0VdIWSZcWtF8gaUNad1uaazhTLU2eb9jMKlcxZwJdwO9GxHxgGXCTpPlp3Z9HxOL0eBAgrVsJ\nLACWA7en+YkB7gCuJz/5/Ly0PlOtuUZe2XeIzq7uwTc2MxtjBg2BiNgVEU+l5QPAZmDWSXa5Crg3\nIjojYhuwFVgqqRmYFBGPRf6rO+8Brh7ybzBELU2N9ARsf+1Q1qWYmZXcKfUJSDqb/KTzj6emz0la\nL+kuSVNS2yxgR8FuO1PbrLTctz1TnmrSzCpZ0SEgqRH4HvCFiNhP/tLOXGAxsAv46nAVJekGSW2S\n2jo6OobrZfs1N+cvkjOzylVUCEiqJR8A346IfwKIiD0R0R0RPcA3gKVp83ZgTsHus1Nbe1ru236C\niLgzIpZExJJcLncqv88pm1BXw6wzxjsEzKwiFTM6SMC3gM0R8bWC9uaCzT4BPJuW1wArJdVLOod8\nB/ATEbEL2C9pWXrNa4D7hun3GJKWpkZfDjKzilRTxDYXAp8GNkh6OrX9IfApSYuBALYDnwGIiI2S\nVgObyI8suikieofe3AjcDYwH1qZH5lpzjTyx7XV6eoKqqsxHrZqZlcygIRARjwD9/WV88CT73Arc\n2k97G7DwVAoshZamBg4f7eHVt95h9pQJWZdjZlYyFX3HcK9jXyTnfgEzqzAOAaD12BfJeW4BM6ss\nDgFgakMdZ0yo9ZmAmVUchwD5qSZbcx4hZGaVxyGQtOQa/ZXSZlZxHAJJa1Mjrx88whsHj2RdiplZ\nyTgEkpam/NdH+JKQmVUSh0DSmpsIeJiomVUWh0Aya8p46mqqfCZgZhXFIZBUV4m50z3LmJlVFodA\ngdamRt8wZmYVxSFQoCXXyI43DnH4qKeaNLPK4BAo0NrUSARse81nA2ZWGRwCBVr8RXJmVmEcAgXm\n5hqQfK+AmVUOh0CBcbXVzJ7iqSbNrHI4BPpozTU6BMysYhQzx/AcST+WtEnSRkmfT+1TJT0k6YX0\nPKVgn1skbZW0RdKlBe0XSNqQ1t2W5houKy25Rra9dpDunsi6FDOzEVfMmUAX8LsRMR9YBtwkaT5w\nM7AuIuYB69LPpHUrgQXAcuB2SdXpte4Aric/+fy8tL6stDY10tnVQ/sb72RdipnZiBs0BCJiV0Q8\nlZYPAJuBWcBVwKq02Srg6rR8FXBvRHRGxDZgK7BUUjMwKSIei4gA7inYp2y0HJtlzJeEzGzsO6U+\nAUlnA+8FHgdmRMSutGo3MCMtzwJ2FOy2M7XNSst928uK5xs2s0pSdAhIagS+B3whIvYXrkv/sh+2\ni+iSbpDUJqmto6NjuF62KFMa6pjWUOczATOrCEWFgKRa8gHw7Yj4p9S8J13iIT3vTe3twJyC3Wen\ntva03Lf9BBFxZ0QsiYgluVyu2N9l2LR4hJCZVYhiRgcJ+BawOSK+VrBqDXBtWr4WuK+gfaWkeknn\nkO8AfiJdOtovaVl6zWsK9ikrLU2eb9jMKkNNEdtcCHwa2CDp6dT2h8CXgdWSrgNeBj4JEBEbJa0G\nNpEfWXRTRPR+I9uNwN3AeGBtepSdllwDbxw6yutvdzKtsT7rcszMRsygIRARjwADjef/yAD73Arc\n2k97G7DwVArMQuuxEUIHHQJmNqb5juF++IvkzKxSOAT6MeuM8YyrrXIImNmY5xDoR1WVmDvdncNm\nNvY5BAbQ2uRhomY29jkEBtDa1Ej7m+/wzhFPNWlmY5dDYAC9ncO+JGRmY5lDYACt/iI5M6sADoEB\nnD19AlXyMFEzG9scAgOor6lm4azJPPx8ab/AzsyslBwCJ3HF+c2s3/kW2187mHUpZmYjwiFwElec\nPxOA+595NeNKzMxGhkPgJGaeMZ73nz2F+9c7BMxsbHIIDGLFopk8v+dttuw+kHUpZmbDziEwiMvf\n00yVYM0z/c5/Y2Y2qjkEBjG9sZ4LW6dz/zO7yM+iaWY2djgEirDi/Jm8su8Q63e+lXUpZmbDyiFQ\nhEsXnklttVjjUUJmNsYUM8fwXZL2Snq2oO2LktolPZ0elxesu0XSVklbJF1a0H6BpA1p3W1pnuFR\nYfL4Wi5+dxMPrH+Vnh5fEjKzsaOYM4G7geX9tP95RCxOjwcBJM0HVgIL0j63S6pO298BXE9+4vl5\nA7xm2bpy8Uz27O/kie37si7FzGzYDBoCEfFToNi/fFcB90ZEZ0RsA7YCSyU1A5Mi4rHI967eA1x9\nukVn4aO/1MT42mrfOGZmY8pQ+gQ+J2l9ulw0JbXNAnYUbLMztc1Ky33bR40JdTV85JeaWPvsbo52\n92RdjpnZsDjdELgDmAssBnYBXx22igBJN0hqk9TW0VE+X+B25aKZ7Dt4hJ9tfS3rUszMhsVphUBE\n7ImI7ojoAb4BLE2r2oE5BZvOTm3tablv+0Cvf2dELImIJblc7nRKHBEXn5tj4rga7n9mV9almJkN\ni9MKgXSNv9cngN6RQ2uAlZLqJZ1DvgP4iYjYBeyXtCyNCroGuG8IdWeivqaa5QvO5F827ubwUU87\naWajXzFDRL8DPAqcK2mnpOuAr6ThnuuBXwX+O0BEbARWA5uAHwI3RUTvX8sbgW+S7yx+EVg73L9M\nKaxYNJMDnV38ZEv5XKYyMztdNYNtEBGf6qf5WyfZ/lbg1n7a24CFp1RdGfpAyzSmNdRx//pXWb7w\nzKzLMTMbEt8xfIpqqqu4/D3NrNu8h4OdXVmXY2Y2JA6B07Bi0UwOH+3hR5v3ZF2KmdmQOAROw5J3\nTaF58jjfOGZmo55D4DRUVYkrzm/m4ec7ePPQkazLMTM7bQ6B07Ri0UyOdgf/vHF31qWYmZ02h8Bp\nes+syZw9bYJvHDOzUc0hcJoksWLRTP7txdfYe+Bw1uWYmZ0Wh8AQrFg0k56AtRt8ScjMRieHwBC8\ne8ZEzjtzomccM7NRyyEwRCsWzeTJl99g5xuHsi7FzOyUOQSGaMX5MwH4wXp3EJvZ6OMQGKKzpk1g\n0ZwzfEnIzEYlh8AwWHF+Mxtf3c+LHW9nXYqZ2SlxCAyDK86fiYS/RsLMRh2HwDA4c/I4lp49lfuf\neZWIyLocM7OiOQSGyZWLZ/Jix0E27zqQdSlmZkVzCAyTyxY2U10ldxCb2ahSzPSSd0naK+nZgrap\nkh6S9EJ6nlKw7hZJWyVtkXRpQfsFaUrKrZJuS3MNjxlTG+q4qHW6LwmZ2ahSzJnA3cDyPm03A+si\nYh6wLv2MpPnASmBB2ud2SdVpnzuA68lPPj+vn9cc9a5cNJP2N9/h33e8mXUpZmZFGTQEIuKnwL4+\nzVcBq9LyKuDqgvZ7I6IzIraRn1R+qaRmYFJEPBb5fybfU7DPmPHxBTOoq6lizdO+JGRmo8Pp9gnM\niIjeW2R3AzPS8ixgR8F2O1PbrLTct31MmTiulg+f28QPNuyiu8eXhMys/A25Yzj9y35Y/+JJukFS\nm6S2jo6O4XzpEbdi0Uw6DnTy+EuvZ12KmdmgTjcE9qRLPKTnvam9HZhTsN3s1Naelvu29ysi7oyI\nJRGxJJfLnWaJ2fjweU001FVz/3pfEjKz8ne6IbAGuDYtXwvcV9C+UlK9pHPIdwA/kS4d7Ze0LI0K\nuqZgnzFlfF01H5s/g7XP7uZIV0/W5ZiZnVQxQ0S/AzwKnCtpp6TrgC8DH5P0AvDR9DMRsRFYDWwC\nfgjcFBHd6aVuBL5JvrP4RWDtMP8uZWPFopm8eegoj2wdXZeyzKzy1Ay2QUR8aoBVHxlg+1uBW/tp\nbwMWnlJ1o9QH5+WYPL6W+5/ZxYfPmzH4DmZmGfEdwyOgrqaKyxaeyb9s3M3ho92D72BmlhGHwAhZ\nsWgmB49086/P7R18YzOzjDgERsiyudOY3ljvr5c2s7LmEBgh1VXiivObWffcXg4cPpp1OWZm/XII\njKAVi5o50tXDA55/2MzKlENgBL3vrCksnnMG/+vBzezYdyjrcszMTuAQGEGS+KtPvRcBv/X3T/nm\nMTMrOw6BETZn6gT+9DcW8czOt/jS2s1Zl2NmdhyHQAlcuuBMfvPCc/jfP9vOD5/dnXU5ZmbHOARK\n5ObLzmPR7Mn83nef4ZXX3T9gZuXBIVAidTVV/PV/fl++f+A7T9HZ5TuJzSx7DoES6u0fWL/zLb70\n4HNZl2Nm5hAotd7+gbv/bTs/fNb3D5hZthwCGfhF/8B69w+YWaYcAhlw/4CZlQuHQEbcP2Bm5cAh\nkCH3D5hZ1oYUApK2S9og6WlJbaltqqSHJL2QnqcUbH+LpK2Stki6dKjFjwU3X3Yei+ac4f4BM8vE\ncJwJ/GpELI6IJennm4F1ETEPWJd+RtJ8YCWwAFgO3C6pehjef1Srq6nir3u/X8j9A2ZWYiNxOegq\nYFVaXgVcXdB+b0R0RsQ28hPOLx2B9x915kydwJ+5f8DMMjDUEAjgR5KelHRDapsREb0XuHcDvTOt\nzwJ2FOy7M7UZ8PEFZ3LdRe4fMLPSqhni/hdFRLukJuAhScf9MzYiQlKc6oumQLkB4KyzzhpiiaPH\nHyw/j7aX3+D3vrue+c2TOWvahKxLMrMxbkhnAhHRnp73At8nf3lnj6RmgPTcO9N6OzCnYPfZqa2/\n170zIpZExJJcLjeUEkcV9w+YWamddghIapA0sXcZ+DjwLLAGuDZtdi1wX1peA6yUVC/pHGAe8MTp\nvv9Y5f4BMyuloVwOmgF8X1Lv6/x9RPxQ0s+B1ZKuA14GPgkQERslrQY2AV3ATRHhf+r2o7d/4FuP\nbGPZ3KksX9icdUlmNkYp4pQv2ZfUkiVLoq2tLesySu5IVw+/8fVHeanjbX7wuQ+6f8DMTomkJwuG\n7g/IdwyXqcL+gZv+3v0DZjYyHAJlrLd/YEP7W1z5Vz+jbfu+rEsyszHGIVDmPr7gTL55zRIOHD7K\nr//to/zBd9fzxsEjWZdlZmOEQ2AU+Oj8GTz0OxfzmQ/N5XtP7eTDX/0Jq9t2UO79OWZW/hwCo0RD\nfQ23XP5LPPDbF9GSa+T3v7ue//T1x3h+z4GsSzOzUcwhMMqcd+YkVn/mV/jKfzyf5/ce4PK//H98\nae1mDh3pyro0MxuFHAKjUFWV+OT75/Cvv3sJv/a+WXz94Zf42Nd+ykOb9mRdmpmNMg6BUWxqQx1f\n+fVF/ONnf4WG+mquv6eN6+9po/3Nd7IuzcxGCYfAGPD+s6fyg9/+IDdfdh6PvPAaH/3qw3z94Rc5\n2t2TdWlmVuYcAmNEbXUVn724hYd+50Nc2DqdL619jitue8T3FpjZSTkExpjZUybwzWuXcOenL/C9\nBWY2qKHOJ2Bl6uMLzuTC1unctu4FvvXINv55024uf08zl7w7xwdap9NY7//0ZuYvkKsIW3Yf4C9+\n9Dw/fb6Dg0e6qa0W7z97Kpecm+OSc5uY19RI+jZYMxsjiv0COYdABTnS1UPby/t4eEsHP9nSwZZ0\no9nMyeO4+NwcF7+7iQtbpzFxXG3GlZrZUDkEbFCvvvkODz/fwU+27OVnW1/n7c4uaqrEkrOncMm5\nTVxybo5zZ0z0WYLZKOQQsFNypKuHp155g59syYfCc7vzZwnNk8dx8btzXHJujgtbp/sswWyUcAjY\nkOx+6zAPP7+Xn2zp4JEXXuNAZxcSnDlpHHOmTuCsgsecqeOZM3UCucZ6nzWYlYmyDQFJy4G/BKqB\nb0bEl0+2vUMge0e7e3jq5Td4fNs+Xn79EDv2HeKVfYfYvf/wcduNq606FgyzpxQExbQJzJkygfF1\n1Rn9BmaVp9gQKOk4QUnVwN8AHwN2Aj+XtCYiNpWyDjs1tdVV/PLcafzy3GnHtR8+2k37m+/wyr4U\nDK/nw+GVfYd49MXXOXjk+NnQpjfWM2vKeM4YX8uk8bVMHFfDpHG1TBpfw8RxtUwaV8Ok8bX5toLl\ncbVVPsMwGyGlHiy+FNgaES8BSLoXuIr85PM2yoyrraYl10hLrvGEdRHBG4eOHguF3pB49a13ePPQ\nEXbsO8T+w0fZ/04XRwb5eouaKqVAyAdDQ10N9bVV1FVXUV9bTV11FXU1VdSnR13Bc3/b9LZXV+m4\nR5VETbWolk5Y119blfIPifwzIOHAslGl1CEwC9hR8PNO4JdLXIOVgCSmNtQxtaGOxXPOGHC7iKCz\nq+dYIOSfj3LgcNdxbQcKlt8+3MXBI10c6eqhs6vn2HNnVzdHuno40t1Dll1dEgiOBYRSQORDI/2c\ntum7DBwLk/Rqx9b/4rV13PsMFDqFzcctoxPaB4qtAV+76MZTMxzxWS4hPBxVPPDbF1FfM7KXUcvy\ntlFJNwA3AJx11lkZV2MjSRLjaqsZV1tN08Thec2I4Gh3cKS7NyBSOBwLix6OdvfQ0xN09QTdEXR3\n559723oi6Ept3T35R29bT6T9evJJ09MTBNATQUT+/XsCgvR8rC2/vnddb3vAsdDqbc8v97ZH+r1O\n3C6O+70Lljnuh/4Wj81MN1BeDhSk/TUPR9/isOR2mYxziWEqRMMSJSdX6hBoB+YU/Dw7tR0nIu4E\n7oR8x3BpSrOxQhJ1NaKupgrqs67GrLyV+gvkfg7Mk3SOpDpgJbCmxDWYmVlS0jOBiOiS9FvAP5Mf\nInpXRGwsZQ1mZvYLJe8TiIgHgQdL/b5mZnYizydgZlbBHAJmZhXMIWBmVsEcAmZmFcwhYGZWwcr+\nq6QldQAvn+bu04HXhrGckeI6h99oqdV1Dr/RUutI1/muiMgNtlHZh8BQSGor5qtUs+Y6h99oqdV1\nDr/RUmu51OnLQWZmFcwhYGZWwcZ6CNyZdQFFcp3Db7TU6jqH32iptSzqHNN9AmZmdnJj/UzAzMxO\nYkyEgKTlkrZI2irp5n7WS9Jtaf16Se/LoMY5kn4saZOkjZI+3882l0h6S9LT6fHHpa4z1bFd0oZU\nQ1s/6zM/nqmOcwuO1dOS9kv6Qp9tMjmmku6StFfSswVtUyU9JOmF9DxlgH1P+nkuQZ1/Kum59N/2\n+5L6nRpusM9JCer8oqT2gv+2lw+wb8mO50lq/YeCOrdLenqAfUt2TI+JiFH9IP+V1C8Cc4E64Blg\nfp9tLgfWkp/xbRnweAZ1NgPvS8sTgef7qfMS4IEyOKbbgeknWZ/58Rzgc7Cb/NjozI8p8CHgfcCz\nBW1fAW5OyzcDfzLA73HSz3MJ6vw4UJOW/6S/Oov5nJSgzi8C/6OIz0XJjudAtfZZ/1Xgj7M+pr2P\nsXAmcGzy+og4AvROXl/oKuCeyHsMOENScymLjIhdEfFUWj4AbCY/5/JolPnx7MdHgBcj4nRvLBxW\nEfFTYF+f5quAVWl5FXB1P7sW83ke0Toj4l8ioiv9+Bj5GQAzNcDxLEZJjyecvFblJ0D+JPCdkazh\nVIyFEOhv8vq+f1yL2aZkJJ0NvBd4vJ/VH0in4WslLShpYb8QwI8kPZnme+6rrI5nspKB/8cqh2MK\nMCMidqXl3cCMfrYpt2P7m+TP+voz2OekFD6X/tveNcDltXI7nh8E9kTECwOsL/kxHQshMKpIagS+\nB3whIvb3Wf0UcFZEnA/8FfB/S11fclFELAYuA26S9KGM6ihKmqr0SuAf+1ldLsf0OJE/9y/roXmS\n/gjoAr49wCZZf07uIH+ZZzGwi/xllnL3KU5+FlDyYzoWQqCYyeuLmuB+pEmqJR8A346If+q7PiL2\nR8TbaflBoFbS9BKXSUS0p+e9wPfJn1IXKovjWeAy4KmI2NN3Rbkc02RP72Wz9Ly3n23K4thK+m/A\nFcB/SYF1giI+JyMqIvZERHdE9ADfGOD9y+J4AkiqAX4N+IeBtsnimI6FEChm8vo1wDVpVMsy4K2C\n0/KSSNcCvwVsjoivDbDNmWk7JC0l/9/n9dJVCZIaJE3sXSbfSfhsn80yP559DPivq3I4pgXWANem\n5WuB+/qpY5+mAAABAklEQVTZppjP84iStBz4feDKiDg0wDbFfE5GVJ9+qE8M8P6ZH88CHwWei4id\n/a3M7JiWshd6pB7kR6s8T34UwB+lts8Cn03LAv4mrd8ALMmgxovIn/6vB55Oj8v71PlbwEbyIxge\nAz6QQZ1z0/s/k2opy+NZUG8D+T/qkwvaMj+m5ENpF3CU/HXo64BpwDrgBeBHwNS07UzgwZN9nktc\n51by19F7P6d/27fOgT4nJa7z79Lnbz35P+zNWR/PgWpN7Xf3fi4Lts3smPY+fMewmVkFGwuXg8zM\n7DQ5BMzMKphDwMysgjkEzMwqmEPAzKyCOQTMzCqYQ8DMrII5BMzMKtj/B/Ts0J5/PTO+AAAAAElF\nTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f78f0395d68>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "## 定义w和b\n",
    "feature_num = 10\n",
    "w_real = np.random.random(feature_num)\n",
    "b_real = np.random.random()\n",
    "\n",
    "## 生成训练数据\n",
    "instance_num = 1000\n",
    "X = np.random.uniform(-100,100,(feature_num, instance_num))\n",
    "y = np.matmul(w_real, X) + b_real\n",
    "y = y + np.random.random(y.shape)\n",
    "\n",
    "## 初始化参数\n",
    "w = np.random.random(feature_num)\n",
    "b = np.random.random()\n",
    "\n",
    "iter_time = 20\n",
    "step_size = 0.0001\n",
    "loss_value = []\n",
    "\n",
    "## 迭代求解\n",
    "for i in range(iter_time):\n",
    "    delta = np.matmul(w, X) + b_real - y\n",
    "    loss_value.append((delta*delta).mean())\n",
    "    w = w - step_size*(np.matmul(delta, X.T))/instance_num\n",
    "    b = b - step_size*delta.mean()\n",
    "plt.clf\n",
    "plt.plot(np.array(loss_value[1:]))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.79842335,  0.42003747,  0.04151777,  0.05549998,  0.71396875,\n",
       "        0.60893848,  0.19337114,  0.42662969,  0.27833712,  0.63035355])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.79855966,  0.41969485,  0.04192018,  0.05457261,  0.7145736 ,\n",
       "        0.60895623,  0.19295485,  0.42620365,  0.27826274,  0.63047412])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w_real"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
